[INFO] cloning repository https://github.com/folkol/llm-inference-benchmark
[INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/folkol/llm-inference-benchmark" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ffolkol%2Fllm-inference-benchmark", kill_on_drop: false }`
[INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ffolkol%2Fllm-inference-benchmark'...
[INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }`
[INFO] [stdout] 2ec705387e920ca8ffd44909fc1d7aa48a3a861b
[INFO] testing folkol/llm-inference-benchmark against try#28f95933dc0727396c07b48d0a8cb5211b7fb472 for pr-154065-2
[INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Ffolkol%2Fllm-inference-benchmark" "/workspace/builds/worker-2-tc2/source", kill_on_drop: false }`
[INFO] [stderr] Cloning into '/workspace/builds/worker-2-tc2/source'...
[INFO] [stderr] done.
[INFO] removed /workspace/builds/worker-2-tc2/source/.cargo/config.toml
[INFO] started tweaking git repo https://github.com/folkol/llm-inference-benchmark
[INFO] finished tweaking git repo https://github.com/folkol/llm-inference-benchmark
[INFO] tweaked toml for git repo https://github.com/folkol/llm-inference-benchmark written to /workspace/builds/worker-2-tc2/source/Cargo.toml
[INFO] validating manifest of git repo https://github.com/folkol/llm-inference-benchmark on toolchain 28f95933dc0727396c07b48d0a8cb5211b7fb472
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+28f95933dc0727396c07b48d0a8cb5211b7fb472" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }`
[INFO] crate git repo https://github.com/folkol/llm-inference-benchmark already has a lockfile, it will not be regenerated
[INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+28f95933dc0727396c07b48d0a8cb5211b7fb472" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }`
[INFO] [stderr]     Blocking waiting for file lock on package cache
[INFO] [stderr]     Blocking waiting for file lock on package cache
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+28f95933dc0727396c07b48d0a8cb5211b7fb472" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }`
[INFO] [stdout] 07db998237f555ec4fdc21f3daf6290c39e3feb44c1241b88d05a816e399dbbc
[INFO] running `Command { std: "docker" "start" "-a" "07db998237f555ec4fdc21f3daf6290c39e3feb44c1241b88d05a816e399dbbc", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "inspect" "07db998237f555ec4fdc21f3daf6290c39e3feb44c1241b88d05a816e399dbbc", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "07db998237f555ec4fdc21f3daf6290c39e3feb44c1241b88d05a816e399dbbc", kill_on_drop: false }`
[INFO] [stdout] 07db998237f555ec4fdc21f3daf6290c39e3feb44c1241b88d05a816e399dbbc
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+28f95933dc0727396c07b48d0a8cb5211b7fb472" "build" "--frozen" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] 785bbc4a15da8b7e032a30cb272ef0d4d683029c8e1c74741090615d0146db7d
[INFO] running `Command { std: "docker" "start" "-a" "785bbc4a15da8b7e032a30cb272ef0d4d683029c8e1c74741090615d0146db7d", kill_on_drop: false }`
[INFO] [stderr]    Compiling libc v0.2.186
[INFO] [stderr]    Compiling smallvec v1.15.1
[INFO] [stderr]    Compiling serde_core v1.0.228
[INFO] [stderr]    Compiling parking_lot_core v0.9.12
[INFO] [stderr]    Compiling pkg-config v0.3.33
[INFO] [stderr]    Compiling itoa v1.0.18
[INFO] [stderr]    Compiling cc v1.2.62
[INFO] [stderr]    Compiling bitflags v2.11.1
[INFO] [stderr]    Compiling writeable v0.6.3
[INFO] [stderr]    Compiling lock_api v0.4.14
[INFO] [stderr]    Compiling litemap v0.8.2
[INFO] [stderr]    Compiling icu_properties_data v2.2.0
[INFO] [stderr]    Compiling crossbeam-utils v0.8.21
[INFO] [stderr]    Compiling syn v2.0.117
[INFO] [stderr]    Compiling indexmap v2.14.0
[INFO] [stderr]    Compiling icu_normalizer_data v2.2.0
[INFO] [stderr]    Compiling generic-array v0.14.7
[INFO] [stderr]    Compiling log v0.4.29
[INFO] [stderr]    Compiling openssl v0.10.79
[INFO] [stderr]    Compiling http v0.2.12
[INFO] [stderr]    Compiling tracing-core v0.1.36
[INFO] [stderr]    Compiling typenum v1.20.0
[INFO] [stderr]    Compiling futures-util v0.3.32
[INFO] [stderr]    Compiling rayon-core v1.13.0
[INFO] [stderr]    Compiling simd-adler32 v0.3.9
[INFO] [stderr]    Compiling crc32fast v1.5.0
[INFO] [stderr]    Compiling futures-channel v0.3.32
[INFO] [stderr]    Compiling tracing v0.1.44
[INFO] [stderr]    Compiling portable-atomic v1.13.1
[INFO] [stderr]    Compiling rustix v1.1.4
[INFO] [stderr]    Compiling zip v2.4.2
[INFO] [stderr]    Compiling bumpalo v3.20.2
[INFO] [stderr]    Compiling miniz_oxide v0.8.9
[INFO] [stderr]    Compiling crossbeam-epoch v0.9.18
[INFO] [stderr]    Compiling anyhow v1.0.102
[INFO] [stderr]    Compiling either v1.15.0
[INFO] [stderr]    Compiling http-body v0.4.6
[INFO] [stderr]    Compiling zopfli v0.8.3
[INFO] [stderr]    Compiling errno v0.3.14
[INFO] [stderr]    Compiling socket2 v0.6.3
[INFO] [stderr]    Compiling mio v1.2.0
[INFO] [stderr]    Compiling socket2 v0.5.10
[INFO] [stderr]    Compiling signal-hook-registry v1.4.8
[INFO] [stderr]    Compiling parking_lot v0.12.5
[INFO] [stderr]    Compiling crossbeam-deque v0.8.6
[INFO] [stderr]    Compiling flate2 v1.1.9
[INFO] [stderr]    Compiling console v0.15.11
[INFO] [stderr]    Compiling filetime v0.2.29
[INFO] [stderr]    Compiling dirs-sys v0.4.1
[INFO] [stderr]    Compiling csv-core v0.1.13
[INFO] [stderr]    Compiling encoding_rs v0.8.35
[INFO] [stderr]    Compiling cpufeatures v0.2.17
[INFO] [stderr]    Compiling colored v2.2.0
[INFO] [stderr]    Compiling dirs v5.0.1
[INFO] [stderr]    Compiling bytesize v1.3.3
[INFO] [stderr]    Compiling openssl-sys v0.9.115
[INFO] [stderr]    Compiling crypto-common v0.1.7
[INFO] [stderr]    Compiling block-buffer v0.10.4
[INFO] [stderr]    Compiling indicatif v0.17.11
[INFO] [stderr]    Compiling digest v0.10.7
[INFO] [stderr]    Compiling rayon v1.12.0
[INFO] [stderr]    Compiling native-tls v0.2.18
[INFO] [stderr]    Compiling sha2 v0.10.9
[INFO] [stderr]    Compiling xattr v1.6.1
[INFO] [stderr]    Compiling tar v0.4.45
[INFO] [stderr]    Compiling serde_json v1.0.149
[INFO] [stderr]    Compiling csv v1.4.0
[INFO] [stderr]    Compiling synstructure v0.13.2
[INFO] [stderr]    Compiling zerofrom-derive v0.1.7
[INFO] [stderr]    Compiling yoke-derive v0.8.2
[INFO] [stderr]    Compiling displaydoc v0.2.5
[INFO] [stderr]    Compiling zerovec-derive v0.11.3
[INFO] [stderr]    Compiling serde_derive v1.0.228
[INFO] [stderr]    Compiling tokio-macros v2.7.0
[INFO] [stderr]    Compiling openssl-macros v0.1.1
[INFO] [stderr]    Compiling thiserror-impl v2.0.18
[INFO] [stderr]    Compiling clap_derive v4.6.1
[INFO] [stderr]    Compiling sysinfo v0.29.11
[INFO] [stderr]    Compiling tokio v1.52.3
[INFO] [stderr]    Compiling zerofrom v0.1.8
[INFO] [stderr]    Compiling yoke v0.8.2
[INFO] [stderr]    Compiling zerotrie v0.2.4
[INFO] [stderr]    Compiling thiserror v2.0.18
[INFO] [stderr]    Compiling zerovec v0.11.6
[INFO] [stderr]    Compiling clap v4.6.1
[INFO] [stderr]    Compiling tinystr v0.8.3
[INFO] [stderr]    Compiling potential_utf v0.1.5
[INFO] [stderr]    Compiling icu_collections v2.2.0
[INFO] [stderr]    Compiling icu_locale_core v2.2.0
[INFO] [stderr]    Compiling serde v1.0.228
[INFO] [stderr]    Compiling serde_spanned v0.6.9
[INFO] [stderr]    Compiling toml_datetime v0.6.11
[INFO] [stderr]    Compiling serde_urlencoded v0.7.1
[INFO] [stderr]    Compiling chrono v0.4.44
[INFO] [stderr]    Compiling toml_edit v0.22.27
[INFO] [stderr]    Compiling icu_provider v2.2.0
[INFO] [stderr]    Compiling icu_normalizer v2.2.0
[INFO] [stderr]    Compiling icu_properties v2.2.0
[INFO] [stderr]    Compiling idna_adapter v1.2.2
[INFO] [stderr]    Compiling idna v1.1.0
[INFO] [stderr]    Compiling toml v0.8.23
[INFO] [stderr]    Compiling url v2.5.8
[INFO] [stderr]    Compiling tokio-util v0.7.18
[INFO] [stderr]    Compiling tokio-native-tls v0.3.1
[INFO] [stderr]    Compiling h2 v0.3.27
[INFO] [stderr]    Compiling hyper v0.14.32
[INFO] [stderr]    Compiling hyper-tls v0.5.0
[INFO] [stderr]    Compiling reqwest v0.11.27
[INFO] [stderr]    Compiling llmb v0.1.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `dev` profile [unoptimized + debuginfo] target(s) in 58.11s
[INFO] running `Command { std: "docker" "inspect" "785bbc4a15da8b7e032a30cb272ef0d4d683029c8e1c74741090615d0146db7d", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "785bbc4a15da8b7e032a30cb272ef0d4d683029c8e1c74741090615d0146db7d", kill_on_drop: false }`
[INFO] [stdout] 785bbc4a15da8b7e032a30cb272ef0d4d683029c8e1c74741090615d0146db7d
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+28f95933dc0727396c07b48d0a8cb5211b7fb472" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }`
[INFO] [stdout] d47cba88ffb3e330ccdb8e31e8a3b01bf8b11c314d045e0323c62aa2136a533c
[INFO] running `Command { std: "docker" "start" "-a" "d47cba88ffb3e330ccdb8e31e8a3b01bf8b11c314d045e0323c62aa2136a533c", kill_on_drop: false }`
[INFO] [stderr]    Compiling llmb v0.1.0 (/opt/rustwide/workdir)
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 3.38s
[INFO] running `Command { std: "docker" "inspect" "d47cba88ffb3e330ccdb8e31e8a3b01bf8b11c314d045e0323c62aa2136a533c", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "d47cba88ffb3e330ccdb8e31e8a3b01bf8b11c314d045e0323c62aa2136a533c", kill_on_drop: false }`
[INFO] [stdout] d47cba88ffb3e330ccdb8e31e8a3b01bf8b11c314d045e0323c62aa2136a533c
[INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-2-tc2/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:d429b63d4308055ea97f60fb1d3dfca48854a00942f1bd2ad806beaf015945ec" "/opt/rustwide/cargo-home/bin/cargo" "+28f95933dc0727396c07b48d0a8cb5211b7fb472" "test" "--frozen", kill_on_drop: false }`
[INFO] [stdout] a9b0067a2b4a6c58d77b691653b4000f1128d63fd36a9526f28049be1dd2dff5
[INFO] running `Command { std: "docker" "start" "-a" "a9b0067a2b4a6c58d77b691653b4000f1128d63fd36a9526f28049be1dd2dff5", kill_on_drop: false }`
[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.27s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/llmb-5be70661ebb40625)
[INFO] [stdout] 
[INFO] [stdout] running 7 tests
[INFO] [stdout] test hardware::tests::report_path_label_matches_known_snapshot_host ... ok
[INFO] [stdout] test hardware::tests::report_path_label_without_gpu ... ok
[INFO] [stdout] test setup::tests::find_asset_prefers_plain_macos_arm64_tarball ... ok
[INFO] [stdout] test setup::tests::find_asset_accepts_zip_and_tar_gz ... ok
[INFO] [stdout] test setup::tests::find_asset_rejects_unrelated_archives ... ok
[INFO] [stderr]      Running unittests src/main.rs (/opt/rustwide/target/debug/deps/llmb-1e822834583d2d76)
[INFO] [stdout] test setup::tests::find_windows_cuda_prefers_newer_explicit_toolkit ... ok
[INFO] [stdout] test setup::tests::find_windows_cudart_matches_llama_zip_toolkit_version ... ok
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.02s
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] running 0 tests
[INFO] [stdout] 
[INFO] [stdout] test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
[INFO] [stdout] 
[INFO] [stderr]      Running tests/e2e.rs (/opt/rustwide/target/debug/deps/e2e-cf1e457b0a50360d)
[INFO] [stdout] 
[INFO] [stdout] running 5 tests
[INFO] [stdout] test test_e2e_same_seed_same_token_count ... FAILED
[INFO] [stdout] test test_e2e_cold_inference_succeeds ... FAILED
[INFO] [stdout] test test_e2e_aggregated_metrics_are_sane ... FAILED
[INFO] [stdout] test test_e2e_tokens_per_sec_plausible ... FAILED
[INFO] [stdout] test test_e2e_different_seeds_both_complete ... FAILED
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_same_seed_same_token_count stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_same_seed_same_token_count' (31) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_same_seed_same_token_count
[INFO] [stdout]              at ./tests/e2e.rs:120:15
[INFO] [stdout]    5: e2e::test_e2e_same_seed_same_token_count::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:119:41
[INFO] [stdout]    6: <e2e::test_e2e_same_seed_same_token_count::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::trace_unsynchronized::<std[d054f6f42beb1baa]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d8691f6af38 - std[d054f6f42beb1baa]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d8691f6af38 - <<std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[5a1c3e9896636f3a]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d8691f7fb0a - <core[5a1c3e9896636f3a]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d8691f7fb0a - core[5a1c3e9896636f3a]::fmt::write
[INFO] [stdout]    6:     0x5d8691f6fc82 - std[d054f6f42beb1baa]::io::default_write_fmt::<alloc[154f689e5dd3bae5]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:622:11
[INFO] [stdout]    7:     0x5d8691f6fc82 - <alloc[154f689e5dd3bae5]::vec::Vec<u8> as std[d054f6f42beb1baa]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:1977:13
[INFO] [stdout]    8:     0x5d8691f48b96 - <std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d8691f48b96 - std[d054f6f42beb1baa]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d8691f633f9 - std[d054f6f42beb1baa]::panicking::default_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d8691eadecc - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   12:     0x5d8691eadecc - test[fad20673b750c10e]::test_main_inner::<test[fad20673b750c10e]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d8691f635b2 - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   14:     0x5d8691f635b2 - std[d054f6f42beb1baa]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:833:13
[INFO] [stdout]   15:     0x5d8691f48c52 - std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:698:13
[INFO] [stdout]   16:     0x5d8691f3ffa9 - std[d054f6f42beb1baa]::sys::backtrace::__rust_end_short_backtrace::<std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d8691f49a5d - __rustc[7ab284efd7f33ea8]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:689:5
[INFO] [stdout]   18:     0x5d8691f8029c - core[5a1c3e9896636f3a]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d8691f80012 - core[5a1c3e9896636f3a]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1872:5
[INFO] [stdout]   20:     0x5d8691ea03f3 - <core[5a1c3e9896636f3a]::result::Result<std[d054f6f42beb1baa]::path::PathBuf, anyhow[7fadc68ac79c0313]::Error>>::expect
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1185:23
[INFO] [stdout]   21:     0x5d8691e9edb9 - e2e[80d7d16eb88faa7e]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d8691e9f4d4 - e2e[80d7d16eb88faa7e]::test_e2e_same_seed_same_token_count
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:120:15
[INFO] [stdout]   23:     0x5d8691e9e177 - e2e[80d7d16eb88faa7e]::test_e2e_same_seed_same_token_count::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:119:41
[INFO] [stdout]   24:     0x5d8691ea0e86 - <e2e[80d7d16eb88faa7e]::test_e2e_same_seed_same_token_count::{closure#0} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d8691ea0fab - <fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d8691ea0fab - test[fad20673b750c10e]::__rust_begin_short_backtrace::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d8691eae825 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   30:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d8691eae825 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d8691eae825 - test[fad20673b750c10e]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d8691ea9384 - test[fad20673b750c10e]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d8691ea9384 - std[d054f6f42beb1baa]::sys::backtrace::__rust_begin_short_backtrace::<test[fad20673b750c10e]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d8691eb18a2 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   39:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::<(), core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d8691eb18a2 - <std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d8691f6a89f - <alloc[154f689e5dd3bae5]::boxed::Box<dyn core[5a1c3e9896636f3a]::ops::function::FnOnce<(), Output = ()> + core[5a1c3e9896636f3a]::marker::Send> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   44:     0x5d8691f6a89f - <std[d054f6f42beb1baa]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   45:     0x79ec4ba83aa4 - <unknown>
[INFO] [stdout]   46:     0x79ec4bb10a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_cold_inference_succeeds stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_cold_inference_succeeds' (28) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_cold_inference_succeeds
[INFO] [stdout]              at ./tests/e2e.rs:101:15
[INFO] [stdout]    5: e2e::test_e2e_cold_inference_succeeds::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:100:38
[INFO] [stdout]    6: <e2e::test_e2e_cold_inference_succeeds::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::trace_unsynchronized::<std[d054f6f42beb1baa]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d8691f6af38 - std[d054f6f42beb1baa]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d8691f6af38 - <<std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[5a1c3e9896636f3a]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d8691f7fb0a - <core[5a1c3e9896636f3a]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d8691f7fb0a - core[5a1c3e9896636f3a]::fmt::write
[INFO] [stdout]    6:     0x5d8691f6fc82 - std[d054f6f42beb1baa]::io::default_write_fmt::<alloc[154f689e5dd3bae5]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:622:11
[INFO] [stdout]    7:     0x5d8691f6fc82 - <alloc[154f689e5dd3bae5]::vec::Vec<u8> as std[d054f6f42beb1baa]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:1977:13
[INFO] [stdout]    8:     0x5d8691f48b96 - <std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d8691f48b96 - std[d054f6f42beb1baa]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d8691f633f9 - std[d054f6f42beb1baa]::panicking::default_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d8691eadecc - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   12:     0x5d8691eadecc - test[fad20673b750c10e]::test_main_inner::<test[fad20673b750c10e]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d8691f635b2 - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   14:     0x5d8691f635b2 - std[d054f6f42beb1baa]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:833:13
[INFO] [stdout]   15:     0x5d8691f48c52 - std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:698:13
[INFO] [stdout]   16:     0x5d8691f3ffa9 - std[d054f6f42beb1baa]::sys::backtrace::__rust_end_short_backtrace::<std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d8691f49a5d - __rustc[7ab284efd7f33ea8]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:689:5
[INFO] [stdout]   18:     0x5d8691f8029c - core[5a1c3e9896636f3a]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d8691f80012 - core[5a1c3e9896636f3a]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1872:5
[INFO] [stdout]   20:     0x5d8691ea03f3 - <core[5a1c3e9896636f3a]::result::Result<std[d054f6f42beb1baa]::path::PathBuf, anyhow[7fadc68ac79c0313]::Error>>::expect
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1185:23
[INFO] [stdout]   21:     0x5d8691e9edb9 - e2e[80d7d16eb88faa7e]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d8691e9ede1 - e2e[80d7d16eb88faa7e]::test_e2e_cold_inference_succeeds
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:101:15
[INFO] [stdout]   23:     0x5d8691e9e117 - e2e[80d7d16eb88faa7e]::test_e2e_cold_inference_succeeds::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:100:38
[INFO] [stdout]   24:     0x5d8691ea0e06 - <e2e[80d7d16eb88faa7e]::test_e2e_cold_inference_succeeds::{closure#0} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d8691ea0fab - <fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d8691ea0fab - test[fad20673b750c10e]::__rust_begin_short_backtrace::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d8691eae825 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   30:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d8691eae825 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d8691eae825 - test[fad20673b750c10e]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d8691ea9384 - test[fad20673b750c10e]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d8691ea9384 - std[d054f6f42beb1baa]::sys::backtrace::__rust_begin_short_backtrace::<test[fad20673b750c10e]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d8691eb18a2 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   39:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::<(), core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d8691eb18a2 - <std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d8691f6a89f - <alloc[154f689e5dd3bae5]::boxed::Box<dyn core[5a1c3e9896636f3a]::ops::function::FnOnce<(), Output = ()> + core[5a1c3e9896636f3a]::marker::Send> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   44:     0x5d8691f6a89f - <std[d054f6f42beb1baa]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   45:     0x79ec4ba83aa4 - <unknown>
[INFO] [stdout]   46:     0x79ec4bb10a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_aggregated_metrics_are_sane stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_aggregated_metrics_are_sane' (27) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_aggregated_metrics_are_sane
[INFO] [stdout]              at ./tests/e2e.rs:158:15
[INFO] [stdout]    5: e2e::test_e2e_aggregated_metrics_are_sane::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:157:42
[INFO] [stdout]    6: <e2e::test_e2e_aggregated_metrics_are_sane::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::trace_unsynchronized::<std[d054f6f42beb1baa]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d8691f6af38 - std[d054f6f42beb1baa]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d8691f6af38 - <<std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[5a1c3e9896636f3a]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d8691f7fb0a - <core[5a1c3e9896636f3a]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d8691f7fb0a - core[5a1c3e9896636f3a]::fmt::write
[INFO] [stdout]    6:     0x5d8691f6fc82 - std[d054f6f42beb1baa]::io::default_write_fmt::<alloc[154f689e5dd3bae5]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:622:11
[INFO] [stdout]    7:     0x5d8691f6fc82 - <alloc[154f689e5dd3bae5]::vec::Vec<u8> as std[d054f6f42beb1baa]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:1977:13
[INFO] [stdout]    8:     0x5d8691f48b96 - <std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d8691f48b96 - std[d054f6f42beb1baa]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d8691f633f9 - std[d054f6f42beb1baa]::panicking::default_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d8691eadecc - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   12:     0x5d8691eadecc - test[fad20673b750c10e]::test_main_inner::<test[fad20673b750c10e]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d8691f635b2 - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   14:     0x5d8691f635b2 - std[d054f6f42beb1baa]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:833:13
[INFO] [stdout]   15:     0x5d8691f48c52 - std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:698:13
[INFO] [stdout]   16:     0x5d8691f3ffa9 - std[d054f6f42beb1baa]::sys::backtrace::__rust_end_short_backtrace::<std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d8691f49a5d - __rustc[7ab284efd7f33ea8]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:689:5
[INFO] [stdout]   18:     0x5d8691f8029c - core[5a1c3e9896636f3a]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d8691f80012 - core[5a1c3e9896636f3a]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1872:5
[INFO] [stdout]   20:     0x5d8691ea03f3 - <core[5a1c3e9896636f3a]::result::Result<std[d054f6f42beb1baa]::path::PathBuf, anyhow[7fadc68ac79c0313]::Error>>::expect
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1185:23
[INFO] [stdout]   21:     0x5d8691e9edb9 - e2e[80d7d16eb88faa7e]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d8691e9f9d1 - e2e[80d7d16eb88faa7e]::test_e2e_aggregated_metrics_are_sane
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:158:15
[INFO] [stdout]   23:     0x5d8691e9e1a7 - e2e[80d7d16eb88faa7e]::test_e2e_aggregated_metrics_are_sane::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:157:42
[INFO] [stdout]   24:     0x5d8691ea0ec6 - <e2e[80d7d16eb88faa7e]::test_e2e_aggregated_metrics_are_sane::{closure#0} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d8691ea0fab - <fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d8691ea0fab - test[fad20673b750c10e]::__rust_begin_short_backtrace::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d8691eae825 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   30:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d8691eae825 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d8691eae825 - test[fad20673b750c10e]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d8691ea9384 - test[fad20673b750c10e]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d8691ea9384 - std[d054f6f42beb1baa]::sys::backtrace::__rust_begin_short_backtrace::<test[fad20673b750c10e]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d8691eb18a2 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   39:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::<(), core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d8691eb18a2 - <std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d8691f6a89f - <alloc[154f689e5dd3bae5]::boxed::Box<dyn core[5a1c3e9896636f3a]::ops::function::FnOnce<(), Output = ()> + core[5a1c3e9896636f3a]::marker::Send> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   44:     0x5d8691f6a89f - <std[d054f6f42beb1baa]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   45:     0x79ec4ba83aa4 - <unknown>
[INFO] [stdout]   46:     0x79ec4bb10a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_tokens_per_sec_plausible stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_tokens_per_sec_plausible' (33) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_tokens_per_sec_plausible
[INFO] [stdout]              at ./tests/e2e.rs:199:15
[INFO] [stdout]    5: e2e::test_e2e_tokens_per_sec_plausible::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:198:39
[INFO] [stdout]    6: <e2e::test_e2e_tokens_per_sec_plausible::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::trace_unsynchronized::<std[d054f6f42beb1baa]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d8691f6af38 - std[d054f6f42beb1baa]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d8691f6af38 - <<std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[5a1c3e9896636f3a]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d8691f7fb0a - <core[5a1c3e9896636f3a]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d8691f7fb0a - core[5a1c3e9896636f3a]::fmt::write
[INFO] [stdout]    6:     0x5d8691f6fc82 - std[d054f6f42beb1baa]::io::default_write_fmt::<alloc[154f689e5dd3bae5]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:622:11
[INFO] [stdout]    7:     0x5d8691f6fc82 - <alloc[154f689e5dd3bae5]::vec::Vec<u8> as std[d054f6f42beb1baa]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:1977:13
[INFO] [stdout]    8:     0x5d8691f48b96 - <std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d8691f48b96 - std[d054f6f42beb1baa]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d8691f633f9 - std[d054f6f42beb1baa]::panicking::default_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d8691eadecc - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   12:     0x5d8691eadecc - test[fad20673b750c10e]::test_main_inner::<test[fad20673b750c10e]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d8691f635b2 - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   14:     0x5d8691f635b2 - std[d054f6f42beb1baa]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:833:13
[INFO] [stdout]   15:     0x5d8691f48c52 - std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:698:13
[INFO] [stdout]   16:     0x5d8691f3ffa9 - std[d054f6f42beb1baa]::sys::backtrace::__rust_end_short_backtrace::<std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d8691f49a5d - __rustc[7ab284efd7f33ea8]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:689:5
[INFO] [stdout]   18:     0x5d8691f8029c - core[5a1c3e9896636f3a]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d8691f80012 - core[5a1c3e9896636f3a]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1872:5
[INFO] [stdout]   20:     0x5d8691ea03f3 - <core[5a1c3e9896636f3a]::result::Result<std[d054f6f42beb1baa]::path::PathBuf, anyhow[7fadc68ac79c0313]::Error>>::expect
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1185:23
[INFO] [stdout]   21:     0x5d8691e9edb9 - e2e[80d7d16eb88faa7e]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d8691e9f121 - e2e[80d7d16eb88faa7e]::test_e2e_tokens_per_sec_plausible
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:199:15
[INFO] [stdout]   23:     0x5d8691e9e147 - e2e[80d7d16eb88faa7e]::test_e2e_tokens_per_sec_plausible::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:198:39
[INFO] [stdout]   24:     0x5d8691ea0e46 - <e2e[80d7d16eb88faa7e]::test_e2e_tokens_per_sec_plausible::{closure#0} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d8691ea0fab - <fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d8691ea0fab - test[fad20673b750c10e]::__rust_begin_short_backtrace::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d8691eae825 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   30:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d8691eae825 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d8691eae825 - test[fad20673b750c10e]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d8691ea9384 - test[fad20673b750c10e]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d8691ea9384 - std[d054f6f42beb1baa]::sys::backtrace::__rust_begin_short_backtrace::<test[fad20673b750c10e]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d8691eb18a2 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   39:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::<(), core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d8691eb18a2 - <std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d8691f6a89f - <alloc[154f689e5dd3bae5]::boxed::Box<dyn core[5a1c3e9896636f3a]::ops::function::FnOnce<(), Output = ()> + core[5a1c3e9896636f3a]::marker::Send> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   44:     0x5d8691f6a89f - <std[d054f6f42beb1baa]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   45:     0x79ec4ba83aa4 - <unknown>
[INFO] [stdout]   46:     0x79ec4bb10a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] ---- test_e2e_different_seeds_both_complete stdout ----
[INFO] [stdout] 
[INFO] [stdout] thread 'test_e2e_different_seeds_both_complete' (29) panicked at tests/e2e.rs:26:10:
[INFO] [stdout] llama-cli not found. Install llama.cpp and add it to PATH.
[INFO] [stdout] Download: https://github.com/ggerganov/llama.cpp/releases: Cannot find 'llama-cli'. Install llama.cpp and ensure it is on PATH, or download prebuilt binaries to /root/.cache/llm-benchmark/llama-bin .
[INFO] [stdout] See: https://github.com/ggerganov/llama.cpp/releases
[INFO] [stdout] 
[INFO] [stdout] Stack backtrace:
[INFO] [stdout]    0: <anyhow::Error>::msg::<alloc::string::String>
[INFO] [stdout]              at /opt/rustwide/cargo-home/registry/src/index.crates.io-1949cf8c6b5b557f/anyhow-1.0.102/src/backtrace.rs:10:14
[INFO] [stdout]    1: llmb::assets::find_exe
[INFO] [stdout]              at ./src/assets.rs:59:5
[INFO] [stdout]    2: llmb::assets::find_llama_cli
[INFO] [stdout]              at ./src/assets.rs:42:5
[INFO] [stdout]    3: e2e::llama_cli
[INFO] [stdout]              at ./tests/e2e.rs:25:5
[INFO] [stdout]    4: e2e::test_e2e_different_seeds_both_complete
[INFO] [stdout]              at ./tests/e2e.rs:141:15
[INFO] [stdout]    5: e2e::test_e2e_different_seeds_both_complete::{closure#0}
[INFO] [stdout]              at ./tests/e2e.rs:140:44
[INFO] [stdout]    6: <e2e::test_e2e_different_seeds_both_complete::{closure#0} as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    7: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]    8: test::__rust_begin_short_backtrace::<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]    9: test::run_test_in_process::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   10: <core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   11: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   12: std::panicking::catch_unwind::<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   13: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure#0}>, core::result::Result<(), alloc::string::String>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   14: test::run_test_in_process
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   15: test::run_test::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   16: test::run_test::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   17: std::sys::backtrace::__rust_begin_short_backtrace::<test::run_test::{closure#1}, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   18: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   19: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   20: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   21: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   22: std::panic::catch_unwind::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   23: std::thread::lifecycle::spawn_unchecked::<test::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   24: <std::thread::lifecycle::spawn_unchecked<test::run_test::{closure#1}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25: <alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output = ()> + core::marker::Send> as core::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   26: <std::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]              at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   27: <unknown>
[INFO] [stdout]   28: clone
[INFO] [stdout] stack backtrace:
[INFO] [stdout]    0:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
[INFO] [stdout]    1:     0x5d8691f6af38 - std[d054f6f42beb1baa]::backtrace_rs::backtrace::trace_unsynchronized::<std[d054f6f42beb1baa]::sys::backtrace::_print_fmt::{closure#1}>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
[INFO] [stdout]    2:     0x5d8691f6af38 - std[d054f6f42beb1baa]::sys::backtrace::_print_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:74:9
[INFO] [stdout]    3:     0x5d8691f6af38 - <<std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[5a1c3e9896636f3a]::fmt::Display>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:44:26
[INFO] [stdout]    4:     0x5d8691f7fb0a - <core[5a1c3e9896636f3a]::fmt::rt::Argument>::fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/fmt/rt.rs:152:76
[INFO] [stdout]    5:     0x5d8691f7fb0a - core[5a1c3e9896636f3a]::fmt::write
[INFO] [stdout]    6:     0x5d8691f6fc82 - std[d054f6f42beb1baa]::io::default_write_fmt::<alloc[154f689e5dd3bae5]::vec::Vec<u8>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:622:11
[INFO] [stdout]    7:     0x5d8691f6fc82 - <alloc[154f689e5dd3bae5]::vec::Vec<u8> as std[d054f6f42beb1baa]::io::Write>::write_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/io/mod.rs:1977:13
[INFO] [stdout]    8:     0x5d8691f48b96 - <std[d054f6f42beb1baa]::sys::backtrace::BacktraceLock>::print
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:47:9
[INFO] [stdout]    9:     0x5d8691f48b96 - std[d054f6f42beb1baa]::panicking::default_hook::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:292:27
[INFO] [stdout]   10:     0x5d8691f633f9 - std[d054f6f42beb1baa]::panicking::default_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:316:9
[INFO] [stdout]   11:     0x5d8691eadecc - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   12:     0x5d8691eadecc - test[fad20673b750c10e]::test_main_inner::<test[fad20673b750c10e]::test_main_static::{closure#0}>::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:155:21
[INFO] [stdout]   13:     0x5d8691f635b2 - <alloc[154f689e5dd3bae5]::boxed::Box<dyn for<'a, 'b> core[5a1c3e9896636f3a]::ops::function::Fn<(&'a std[d054f6f42beb1baa]::panic::PanicHookInfo<'b>,), Output = ()> + core[5a1c3e9896636f3a]::marker::Send + core[5a1c3e9896636f3a]::marker::Sync> as core[5a1c3e9896636f3a]::ops::function::Fn<(&std[d054f6f42beb1baa]::panic::PanicHookInfo,)>>::call
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2291:9
[INFO] [stdout]   14:     0x5d8691f635b2 - std[d054f6f42beb1baa]::panicking::panic_with_hook
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:833:13
[INFO] [stdout]   15:     0x5d8691f48c52 - std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:698:13
[INFO] [stdout]   16:     0x5d8691f3ffa9 - std[d054f6f42beb1baa]::sys::backtrace::__rust_end_short_backtrace::<std[d054f6f42beb1baa]::panicking::panic_handler::{closure#0}, !>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:182:18
[INFO] [stdout]   17:     0x5d8691f49a5d - __rustc[7ab284efd7f33ea8]::rust_begin_unwind
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:689:5
[INFO] [stdout]   18:     0x5d8691f8029c - core[5a1c3e9896636f3a]::panicking::panic_fmt
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panicking.rs:80:14
[INFO] [stdout]   19:     0x5d8691f80012 - core[5a1c3e9896636f3a]::result::unwrap_failed
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1872:5
[INFO] [stdout]   20:     0x5d8691ea03f3 - <core[5a1c3e9896636f3a]::result::Result<std[d054f6f42beb1baa]::path::PathBuf, anyhow[7fadc68ac79c0313]::Error>>::expect
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/result.rs:1185:23
[INFO] [stdout]   21:     0x5d8691e9edb9 - e2e[80d7d16eb88faa7e]::llama_cli
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:26:10
[INFO] [stdout]   22:     0x5d8691e9ff34 - e2e[80d7d16eb88faa7e]::test_e2e_different_seeds_both_complete
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:141:15
[INFO] [stdout]   23:     0x5d8691e9e1d7 - e2e[80d7d16eb88faa7e]::test_e2e_different_seeds_both_complete::{closure#0}
[INFO] [stdout]                                at /opt/rustwide/workdir/tests/e2e.rs:140:44
[INFO] [stdout]   24:     0x5d8691ea0f06 - <e2e[80d7d16eb88faa7e]::test_e2e_different_seeds_both_complete::{closure#0} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   25:     0x5d8691ea0fab - <fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   26:     0x5d8691ea0fab - test[fad20673b750c10e]::__rust_begin_short_backtrace::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, fn() -> core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:724:18
[INFO] [stdout]   27:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:74
[INFO] [stdout]   28:     0x5d8691eae825 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   29:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   30:     0x5d8691eae825 - std[d054f6f42beb1baa]::panicking::catch_unwind::<core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>, core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   31:     0x5d8691eae825 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<test[fad20673b750c10e]::run_test_in_process::{closure#0}>, core[5a1c3e9896636f3a]::result::Result<(), alloc[154f689e5dd3bae5]::string::String>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   32:     0x5d8691eae825 - test[fad20673b750c10e]::run_test_in_process
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:747:27
[INFO] [stdout]   33:     0x5d8691eae825 - test[fad20673b750c10e]::run_test::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:668:43
[INFO] [stdout]   34:     0x5d8691ea9384 - test[fad20673b750c10e]::run_test::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/test/src/lib.rs:698:41
[INFO] [stdout]   35:     0x5d8691ea9384 - std[d054f6f42beb1baa]::sys::backtrace::__rust_begin_short_backtrace::<test[fad20673b750c10e]::run_test::{closure#1}, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/backtrace.rs:166:18
[INFO] [stdout]   36:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:70:13
[INFO] [stdout]   37:     0x5d8691eb18a2 - <core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/panic/unwind_safe.rs:275:9
[INFO] [stdout]   38:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::do_call::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:581:40
[INFO] [stdout]   39:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panicking::catch_unwind::<(), core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panicking.rs:544:19
[INFO] [stdout]   40:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::panic::catch_unwind::<core[5a1c3e9896636f3a]::panic::unwind_safe::AssertUnwindSafe<std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}::{closure#0}>, ()>
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/panic.rs:359:14
[INFO] [stdout]   41:     0x5d8691eb18a2 - std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked::<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/thread/lifecycle.rs:68:26
[INFO] [stdout]   42:     0x5d8691eb18a2 - <std[d054f6f42beb1baa]::thread::lifecycle::spawn_unchecked<test[fad20673b750c10e]::run_test::{closure#1}, ()>::{closure#1} as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/core/src/ops/function.rs:250:5
[INFO] [stdout]   43:     0x5d8691f6a89f - <alloc[154f689e5dd3bae5]::boxed::Box<dyn core[5a1c3e9896636f3a]::ops::function::FnOnce<(), Output = ()> + core[5a1c3e9896636f3a]::marker::Send> as core[5a1c3e9896636f3a]::ops::function::FnOnce<()>>::call_once
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/alloc/src/boxed.rs:2277:9
[INFO] [stdout]   44:     0x5d8691f6a89f - <std[d054f6f42beb1baa]::sys::thread::unix::Thread>::new::thread_start
[INFO] [stdout]                                at /rustc/28f95933dc0727396c07b48d0a8cb5211b7fb472/library/std/src/sys/thread/unix.rs:118:17
[INFO] [stdout]   45:     0x79ec4ba83aa4 - <unknown>
[INFO] [stdout]   46:     0x79ec4bb10a64 - clone
[INFO] [stdout]   47:                0x0 - <unknown>
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout]     test_e2e_aggregated_metrics_are_sane
[INFO] [stdout]     test_e2e_cold_inference_succeeds
[INFO] [stdout]     test_e2e_different_seeds_both_complete
[INFO] [stdout]     test_e2e_same_seed_same_token_count
[INFO] [stdout]     test_e2e_tokens_per_sec_plausible
[INFO] [stdout] 
[INFO] [stdout] test result: FAILED. 0 passed; 5 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.12s
[INFO] [stdout] 
[INFO] [stderr] error: test failed, to rerun pass `--test e2e`
[INFO] running `Command { std: "docker" "inspect" "a9b0067a2b4a6c58d77b691653b4000f1128d63fd36a9526f28049be1dd2dff5", kill_on_drop: false }`
[INFO] running `Command { std: "docker" "rm" "-f" "a9b0067a2b4a6c58d77b691653b4000f1128d63fd36a9526f28049be1dd2dff5", kill_on_drop: false }`
[INFO] [stdout] a9b0067a2b4a6c58d77b691653b4000f1128d63fd36a9526f28049be1dd2dff5
